Repository navigation
6d: PI goes 6D-only; Eva and Human gain the continuous-rotation modes - #654
Conversation
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| transform_list: | ||
| _target_: egomimic.rldb.embodiment.eva.Eva.get_transform_list | ||
| mode: cartesian | ||
| mode: cartesian_wristframe_6d |
There was a problem hiding this comment.
eval_pi evaluator still uses wrist-frame revert lists built for 14D or 12D YPR data. During validation, SplitKeys raises on the width mismatch, so camera-frame metrics and video cannot be produced. The new 6D revert lists need to be wired into the evaluator.
| mode: cartesian | ||
| # Wrist-relative actions: each arm's chunk in its current EEF frame, | ||
| # continuous 6D rotation (20-D); the proprio ee_pose stays cam frame. | ||
| mode: cartesian_wristframe_6d |
There was a problem hiding this comment.
eval_hpt evaluator sends them to the Cartesian trajectory visualizer without reverting them. That visualizer accepts only 12D or 14D actions, so validation video generation raises. The migrated Human data has the same problem with its 18D actions.
| self.transform_list = Eva.get_transform_list( | ||
| mode="cartesian_wristframe_ypr", allow_legacy_rotation=True | ||
| ) |
There was a problem hiding this comment.
PolicyRollout loads a checkpoint trained on the new 6D Eva data, this unconditional legacy transform supplies 14D YPR observations where the checkpoint expects 20D 6D proprio. The rollout path also handles predicted actions as 7D YPR blocks. As a result, a new checkpoint cannot produce usable robot commands through this path; the input transform and output decoding need to match its rotation layout.
3ce0cef to
1a00f2a
Compare
d987c91 to
1eab436
Compare
1a00f2a to
f885a7d
Compare
1eab436 to
823b4dc
Compare
The pi0.5 converters stop doing rotation math. Actions now reach the algo
already normalized and already in their model-native layout, so to32_norm_6d /
from32_norm_6d are a pure rearrange into (and out of) openpi's 32-slot block
layout: RobotBimanual's native 20D [L xyz 6d g | R xyz 6d g] IS that layout and
only needs padding, HumanBimanual's 18D inserts a zero gripper slot per arm.
BaseActionConverter raises on both, so an unregistered embodiment fails loudly.
The four single-arm converters (RobotLeft/Right, HumanLeft/Right) are deleted
along with PI._extract_xyz, which existed for the cam-frame MSE. PI's loss
reduction and sample post-processing move into _reduce_loss and
_postprocess_sampled_actions.
The ypr->6D conversion is now a data transform. Eva gains cartesian_6d and
cartesian_wristframe_6d, Human the same pair, each 6D-encoding the action AND
the proprio ee_pose: normalized YPR saturates yaw/roll at +-pi (wraparound), so
per-dim normalization is only meaningful on the continuous representation.
get_revert_transform_list / the _build_*_revert_6d{,_wristframe}_ lists map a
prediction batch back to cam-frame ypr for the viz overlay.
Eva defaults to cartesian_wristframe_6d (eva, cotrain_pi_base, industry_eva_pi;
cotrain_pi_lang_wrist now composes the same as cotrain_pi_lang). Human data
configs move to cartesian_6d with pad_proprio_gripper, which pads the proprio
ee_pose 18 -> 20 (zero grip slots at 9/19) so the pi0.5 State: prompt bins line
up positionally with the robot 20D layout; UnpadGripperZeros inverts it in the
revert lists. HPT dims follow: eva 14 -> 20, human 12 -> proprio 20 / action 18.
With the 6D modes in place, a discontinuous rotation stops being a legal
training target. get_transform_list raises on the cartesian ypr modes and on
Eva's cartesian_wristframe_quat unless the caller passes an explicit
allow_legacy_rotation=True, which only the data/rollout boundary does (the
viz tutorial, rollout.py's legacy-checkpoint path, and the fixtures that use a
cartesian pipeline as scaffolding). test_data_configs_compose asserts that no
shipped data config selects such a mode or sets the opt-out, so the guard
cannot be bypassed from a config. The five *_pickplace_qwen* data configs were
the only ones still on Euler rotations; they and the two hpt_bc_pickplace_qwen
model configs paired with them are deleted. The keypoint modes keep their ypr
and quat variants for now -- the keypoints_*_6d modes do not exist yet.
Also here, because they are the same widening: SplitKeys rejects a layout whose
widths do not sum to the input (an evaluator revert list built for 12-dim ypr
fed an 18/20-dim 6D batch used to slice silently), and vendor-tagged human
embodiment names (MECKA_/SCALE_/ARIA_/LIGHTWHEEL_*) alias onto HUMAN_* through
canonical_embodiment_name, so pins and embodiment checks accept the legacy
metadata. Robot (EVA_*) names are never aliased.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VBmhpVy7ZcMtsBJWh38cJp
823b4dc to
8e66d90
Compare
The pi0.5 converters stop doing rotation math. Actions now reach the algo
already normalized and already in their model-native layout, so to32_norm_6d /
from32_norm_6d are a pure rearrange into (and out of) openpi's 32-slot block
layout: RobotBimanual's native 20D [L xyz 6d g | R xyz 6d g] IS that layout and
only needs padding, HumanBimanual's 18D inserts a zero gripper slot per arm.
BaseActionConverter raises on both, so an unregistered embodiment fails loudly.
The four single-arm converters (RobotLeft/Right, HumanLeft/Right) are deleted
along with PI._extract_xyz, which existed for the cam-frame MSE. PI's loss
reduction and sample post-processing move into _reduce_loss and
_postprocess_sampled_actions.
The ypr->6D conversion is now a data transform. Eva gains cartesian_6d and
cartesian_wristframe_6d, Human the same pair, each 6D-encoding the action AND
the proprio ee_pose: normalized YPR saturates yaw/roll at +-pi (wraparound), so
per-dim normalization is only meaningful on the continuous representation.
get_revert_transform_list / the build*_revert_6d{,wristframe} lists map a
prediction batch back to cam-frame ypr for the viz overlay.
Eva defaults to cartesian_wristframe_6d (eva, cotrain_pi_base, industry_eva_pi;
cotrain_pi_lang_wrist now composes the same as cotrain_pi_lang). Human data
configs move to cartesian_6d with pad_proprio_gripper, which pads the proprio
ee_pose 18 -> 20 (zero grip slots at 9/19) so the pi0.5 State: prompt bins line
up positionally with the robot 20D layout; UnpadGripperZeros inverts it in the
revert lists. HPT dims follow: eva 14 -> 20, human 12 -> proprio 20 / action 18.
Also here, because they are the same widening: SplitKeys rejects a layout whose
widths do not sum to the input (an evaluator revert list built for 12-dim ypr
fed an 18/20-dim 6D batch used to slice silently), and vendor-tagged human
embodiment names (MECKA_/SCALE_/ARIA_/LIGHTWHEEL_) alias onto HUMAN_ through
canonical_embodiment_name, so pins and embodiment checks accept the legacy
metadata. Robot (EVA_*) names are never aliased.
Co-Authored-By: Claude Opus 5 (1M context) [email protected]
🤖 Generated with Claude Code